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Advanced  Prosthetic  Gait  Training  Tool 


INTRODUCTION 

The  objective  of  our  study  is  to  produce  a  computer-based  Advanced  Prosthetic  Gait  Training  Tool  to  aid 
in  professional  development  for  clinicians  and  prosthetists  at  military  treatment  facilities  providing  care 
for  wounded  service  members.  The  effort  will  ultimately  provide  practitioners  at  all  clinics  and  hospitals 
with  access  to  advanced,  computer-based  gait  analysis  tools  that  are  currently  available  at  only  a  few 
state-of-the-art  gait  laboratories.  The  tool  will  aid  in  the  training  of  service  providers,  ultimately 
improving  the  level  of  care  they  provide  to  wounded  veterans. 

Due  to  the  wide  variety  of  injuries  suffered  by  military  personnel  and  the  wide  range  of  medical 
interventions  employed  to  improve  the  standard  of  living  for  patients,  it  is  very  challenging  to  expose 
medical  practitioners  to  a  comprehensive  set  of  human  subjects  to  support  their  training  in  gait 
analysis.  Under  this  effort,  an  extensive  set  of  archived  motion  capture  data  representing  gait  patterns 
for  wounded  service  members  with  varying  challenges  will  be  harvested  from  the  Military  Performance 
Laboratory  at  the  Center  for  the  Intrepid  at  Brooke  Army  Medical  Center  in  San  Antonio. 

To  ensure  confidentiality  for  these  service  members,  the  gait  patterns  will  be  imposed  on  a  digital 
human  model,  referred  to  as  Santos™.  This  computer-based  model  provides  additional  benefit  to 
trainees,  allowing  for  the  repeated  virtual  playback  of  motion  capture  sequences  that  can  be  viewed 
from  any  angle.  Options  will  also  be  provided  to  allow  trainees  to  view  clothed  or  unclothed  avatars, 
stick  figures,  or  even  skeletal  models  to  support  their  analyses.  The  system  will  also  allow  trainees  to 
isolate  specific  parts  of  the  anatomical  model  for  detailed  analysis.  The  trainees  will  enter  their 
assessments  of  observed  gait  deviations,  which  will  be  scored  against  standard  measures  prepared  by 
experts  in  the  field.  Based  on  the  accuracy  of  the  trainees'  responses,  the  system  will  provide 
remediation.  When  fully  developed,  this  system  will  provide  a  comprehensive  training  experience, 
allowing  practitioners  to  benefit  from  a  broad  array  of  patient  data  previously  collected  by  the  US  Army, 
thus  bridging  a  critical  gap  in  current  medical  training  practices.  The  system  will  be  developed  to 
accommodate  additional  sequences  captured  over  time,  thus  offering  an  extensible,  distributable,  and 
sustainable  training  library. 

In  Year  1  of  the  effort,  significant  work  was  completed  at  the  University  of  Iowa  Center  for  Computer- 
Aided  Design  (CCAD),  the  University  of  Iowa  Orthopedic  Gait  Analysis  Laboratory  (OGAL),  and  the 
Military  Performance  Laboratory  (MPL).  A  representative  set  of  motion  capture  sequences  was  provided 
by  MPL  to  CCAD  and  OGAL.  CCAD's  work  focused  on  imposing  these  sequences  on  the  Santos  digital 
human  avatar.  An  initial  user  interface  for  the  training  application  was  also  developed.  These  data  were 
then  provided  to  researchers  from  OGAL  and  MPL  to  support  an  assessment  of  the  ability  of  trained 
clinicians  to  observe  and  accurately  identify  gait  deviations  in  the  target  environment.  Researchers  at 
OGAL  also  embarked  on  a  program  to  develop  a  web-based  questionnaire  using  the  Santos  software. 
This  questionnaire  will  be  sent  to  experts  in  the  field  of  gait  analysis.  The  primary  goal  of  the 
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questionnaire  is  to  identify  the  sensitivity  of  gait  experts  to  detect  variations  in  gait  for  different  severity 
levels  of  the  patients.  In  addition,  the  differences  in  ability  to  detect  variations  in  gait  conditions  for 
skinned  avatar  vs.  line-skeletal  avatar,  concurrent  (side-by-side)  image  representation  vs.  consecutive 
(one  after  the  other)  image  representation,  and  image  vs.  movie  representation  was  also  studied.  The 
information  gathered  from  this  approach  will  be  used  to  identify  a  set  of  gait  profiles  and  to  develop  the 
training  and  evaluation  questionnaire. 

During  Year  1  of  Phase  2  effort,  the  initial  interface  was  improved  further  with  feedback  from  gait 
experts  at  OGAL  to  incorporate  tutorial  options  to  best  train  a  new  clinical  student  in  gait  deviations.  For 
most  of  Year  1  of  phase  2,  MPL  was  responding  to  queries  and  waiting  for  IRB  approval  to  collect  the 
data  necessary  to  go  into  the  software.  However,  OGAL  members  created  a  couple  preliminary  tutorials 
using  dummy  data  to  try  out  various  ideas  in  training.  These  tutorials  served  as  reference  for  VSR  to 
incorporate  the  interface  changes  and  develop  additional  capabilities  to  enhance  learner  interaction 
with  the  software.  In  the  sections  below,  these  training  ideas  and  the  development  of  the  tutorial  are 
discussed. 

The  current  base  effort  built  upon  the  Phase  1  demonstration  of  the  feasibility  of  developing  a 
curriculum  centered  around  motion-capture  data  collected  at  MPL  within  the  Santos  software.  The 
three  main  components  of  the  project  are:  1)  motion  capture  investigations;  2)  gait  deviation  ratings; 
and  3)  software  design.  The  team  was  able  to  progress  on  two  of  these  three  main  components:  the  gait 
deviation  ratings  and  the  software  design. 

Gait  Deviation  Ratings 

We  have  developed  an  assessment  to  examine  general  issues  related  to  the  ability  of  the  user  to 
observe  differences  in  the  walking  pattern.  We  have  created  a  module  to  assess  Visual  Sensitivity  - 
Module  1.  We  have  also  generated  two  modules  to  train  individuals  to  distinguish  gait  deviations: 
Observational  Gait  Analysis  :  Module  1  -  Trunk  Motion  and  Observational  Gait  Analysis  :  Module  2  - 
Lower  Limb  Motion. 

Visual  Sensitivity  (VS)  -  Module  1 

[Link  to  the  module:  https://uiowa.qualtrics.com/SE/?SID=SV  eR5vVCSaSKMASax1 

The  goal  of  the  first  VS  Module  is  to  assess  the  sensitivity  of  observers  in  visually  detecting  differences  in 
static  images  of  the  knee  joint,  represented  at  various  times  during  the  walking  cycle.  To  accomplish 
this,  two  images  of  the  knee  are  presented  with  approximately  0,  5,  or  10  degrees  of  difference.  A 
secondary  goal  of  this  module  is  to  determine  if  the  ability  to  detect  differences  was  affected  by  the 
image  presentation.  To  accomplish  this,  both  stick  figure  representations  and  avatar  representations 
were  used.  Figure  1  below  shows  an  example  of  the  images. 
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Avatar  Form 


Stick  Figure  Form 

Figure  1:  Examples  of  the  type  of  visual  comparisons  that  the  observer  was  asked  to  judge. 

The  users  were  asked  to  rate  their  confidence  in  judging  the  differences  between  the  two  modules 
(Figure  2). 


0  1-  Definitely  no  difference 
0  2-  Probably  no  difference 
©  3-  Unsure  of  a  difference 
0  4-  Probably  a  difference 
5-  Definitely  a  difference 


Figure  2:  Scoring  system  used  by  observers  to  represent  their  ability  to  detect  differences  between  the  two  images. 


Data  from  20  practicing  physical  therapist  has  been  collected  and  is  currently  being  analyzed. 
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Observational  Gait  Analysis  Modules  -  Phase  I 


A  series  of  OGA  Modules  for  Phase  I  are  being  generated  with  the  goal  of  honing  the  observational  skills 
of  the  users.  Modules  in  this  phase  will  focus  on  the  ability  of  the  user  to  discriminate  between  a  more 
ideal  walking  pattern  and  non-specific  gait  deviation.  The  approach  taken  is  to  segment  the  process  by 
focusing  attention  on  the  motion  of  the  trunk  and  the  motion  of  one  lower  limb  before  trying  to 
examine  total  body  motion.  The  body  representations  for  these  initial  modules  are  stick  figures. 

Observational  Gait  Analysis  (OGA)  Module  1:  -  Trunk  Module 

[Link  to  this  module:  https://uiowa.qualtrics.com/5E/7SI  D-SV  7P0WvWVTII  LSHwFl 

This  module  is  focused  on  distinguishing  deviations  in  movements  of  the  head  and  trunk  during  walking. 
The  observer  is  asked  to  consider  stick  figure  representations  of  motions  of  the  head,  arms,  and  trunk 
(HAT)  when  viewed  from  the  front  (frontal  plane)  and  the  side  (sagittal  plane)  relative  to  what  is 
considered  a  more  ideal  pattern  of  motion  (Figure  3). 


Figure  3:  Frontal  and  sagittal  plane  stick  figure  representations  of  the  trunk,  head,  and  arms. 

The  observer  is  shown  a  video  clip  representing  a  normal  movement  pattern  of  the  HAT  and  then  asked 
to  judge  potential  deviations  in  the  movement  of  the  HAT. 
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Close  to  Ideal 
Apparent  Deviations 

Not  Ideal,  but  cannot  distinguish  the  specific  deviations 

Trunk  Motion: 

Items 

R.  Side 
L.  Side 
L.  &  R.  Side 
Front 
Back 

Front  &  Back 


Figure  4:  If  the  user  identifies  that  there  is  a  deviation,  then  they  are  asked  to  judge  the  nature  of  the  deviation. 

Observers  are  then  provided  with  feedback  showing  the  pattern  of  motion  relative  to  a  vertical  axis  and 
comparing  the  current  pattern  to  a  more  ideal  pattern  (Figure  5).  Text  is  also  provided  that  identifies 
the  more  obvious  deviations. 


AJ 

l _ a 

1 

-  *  m 

^  A 

Ideal 

Figure  5:  Once  the  observer  has  decided  if  and  what  the  deviations  in  FIAT  motion  are,  visual  feedback  is  provided  showing  the 
pattern  of  motion  relative  to  the  vertical  orientation  and  relative  to  a  more  ideal  pattern. 

Thus  far,  we  have  distributed  a  draft  of  this  module  to  15  students  in  an  effort  to  get  feedback  on  the 
organization  and  presentation  of  the  material. 
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OGA  Module  2:  Single  Lower  Limb  Module 

[Link  to  this  modulef  https://uiowa.gualtrics.com/SE/7SI  D-SV  88FI8Nc04vtXxkll 

This  module  is  focused  on  distinguishing  deviations  in  movement  of  the  lower  limb  during  walking.  The 
observer  is  asked  to  consider  stick  figure  representations  of  motions  of  the  lower  limb  when  viewed 
from  the  front  (frontal  plane)  and  the  side  (sagittal  plane)  relative  to  what  is  considered  a  more  ideal 
pattern  of  motion  (Figure  6). 


Frontal 

Sagittal 

PELVIS 

A 

* 

|  Left  Side 

• 

♦  L.  KNEE 

I 

\ 

•  V 

I 

*  1 

o  i 

•  *  • 

^  L.  FOOT  # 

■  w 

Figure  6:  The  lower  limb  module  represents  the  movement  of  both  lower  limbs  during  walking  from  the  frontal  and  sagittal 
perspectives. 

Observers  are  asked  to  focus  on  four  critical  times  during  the  walking  cycle  and  to  detect  differences 
from  the  ideal  (Figure  7). 
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Figure  7:  The  position  of  the  lower  limb  at  four  critical  times  during  the  walking  cycle  as  seen  from  two  perspectives. 

Observers  will  initially  be  asked  to  determine  if  they  perceive  a  deviation.  If  they  have  identified  a 
deviant  pattern,  they  will  be  asked  to  attribute  the  deviation  to  specific  issues  in  the  frontal  and  sagittal 
planes  (Figures  8  &  9). 
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Frontal  Plane  Deviations 

Base  of  Support 

Close  to  Ideal 
Too  Wide 
Too  Narrow 

Lower  Lunb  Stance  Alignment  (Clioose  all  that  apply) 

D  Close  to  Ideal 

Knees  In  (Genu  valgus) 

L  I  Knees  Out  (Genu  varus) 

External  Rotation 
Internal  Rotation 

Lower  Lunb  Swing  Alignment  (Choose  all  that  apply) 

Ll  Close  to  Ideal 
D  Circumduction 
Abduction 
Adduction 

1  !  External  Rotation 
—  Internal  Rotation 


Figure  8:  Possible  lower  limb  deviations  observed  in  the  frontal  plane. 


Sagittal  Plane  Deviations 
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Figure  9:  Possible  lower  limb  deviations  observed  in  the  sagittal  plane. 


Once  the  user  has  registered  an  assessment,  visual  and  text  feedback  is  provided  that  helps  to  identify 
the  more  obvious  deviations  (Figure  10). 


9 


Figure  10:  Visual  feedback  provides  information  on  the  ideal  pattern  at  specific  time  in  the  gait  cycle,  as  well  as  a  side-by-side 
comparison  of  the  gait  pattern  relative  to  the  ideal  pattern. 

This  module  has  been  distributed  to  project  personnel  for  feedback.  We  are  currently  revising  it  and 
plan  to  send  it  out  for  additional  feedback. 

Goals 

Our  immediate  goals  are  to  complete  modification,  development,  and  testing  of  the  modules  in  Phase  I. 
We  are  also  in  the  process  of  integrating  these  modules  into  the  environment  generated  by  CCAD. 

In  Phase  II,  we  will  develop  the  training  modules  specific  to  the  gait  patterns  of  individuals  with 
amputations,  based  on  the  gait  data  collected  at  MPL. 

Prosthetic  Gait  Software 

The  functionality  and  visual  appeal  of  the  Prosthetic  Gait  software  has  improved  significantly  over  the 
past  year.  The  layout  and  usage  of  the  software  has  been  developed  into  a  more  intuitive  and  user- 


10 


friendly  format,  and  the  accompanying  quiz  window  has  been  remodeled  to  feature  many  new 
capabilities. 

With  the  default  startup  of  Prosthetic  Gait,  the  Santos  avatar  is  displayed  in  the  center  of  the  screen 
with  50%  opacity  and  his  color-coded  line  body  skeleton  shown.  The  avatar  is  initially  loaded  with  a 
control  MoCap  file.  The  main  toolbar,  including  access  to  the  slide-out  MoCap  library,  the  playback 
tools,  and  the  slide-out  additional  tools/settings,  is  positioned  at  the  top  of  the  screen.  Minimized  by 
default,  the  quiz  window  makes  up  the  bottom  of  the  screen. 

By  clicking  the  MoCap  library  arrow  in  the  top  left  corner,  folders  containing  MoCap  files  slide  into  view. 
Tooltips  containing  each  folder's  category  can  be  seen  by  hovering  the  mouse  over  each  folder.  The 
folders  are  currently  organized  into  three  categories:  Control,  Lower  Body,  and  Upper  Body.  Opening  a 
folder  displays  either  the  sub-folders  inside  of  each  category,  if  any  exist,  or  the  motion  capture  files.  For 
instance,  if  the  Lower  Body  folder  is  clicked,  two  more  folders  (Running  and  Walking)  pop  up.  Selecting 
either  of  these  folders  brings  up  the  corresponding  MoCap  files  associated  with  lower  body  gait  issues. 
The  avatar  is  loaded  with  the  appropriate  data  upon  clicking  a  specific  motion  file.  After  selecting  a  new 
MoCap  file,  an  icon  is  displayed  on  the  respective  file  to  enhance  ease  of  user  navigation  within  the 
folders,  as  seen  in  Figure  11. 

MoCap  files  can  be  played  on  the  avatar  through  the  use  of  the  playback  tools.  The  toolbar  consists  of  a 
play  button,  pause  button,  playback  slider,  running  time,  and  a  play  speed  dropdown  button.  The 
playback  slider  allows  for  the  user  to  manually  select  a  frame  of  motion,  and  the  time  updates  with 
sliding  as  well.  The  playback  speed  dropdown  can  be  used  to  view  motion  at  various  speeds  (x2.0,  xl.O, 
x0.75,  x0.5,  x0.25).  Although  the  playback  speed  functionality  is  not  currently  working,  it  will  be 
corrected  in  the  future. 
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Figure  11:  The  Prosthetic  Gait  software  after  a  MoCap  file  has  been  selected  from  the  slide-out  library.  Additional  tools  have 
also  been  brought  into  view  using  the  arrow  on  the  main  toolbar,  which  also  includes  access  to  the  playback  tools. 

Additional  tools  and  settings  can  also  slide  in  and  out  of  view  using  the  arrow  in  the  top  right  corner. 
Tooltips  display  the  capabilities  of  the  buttons,  which  include:  Show/Hide  Skeleton,  Show/Hide  Control 
Avatar,  Free  Camera  View,  Left/Right  View,  Front/Back  View,  Camera  Follow  On/Off,  and  MoCap 
Offsets.  Clicking  the  Show/Flide  Skeleton  button  toggles  between  an  almost  transparent  avatar  with  line 
skeleton,  a  fully  opaque  avatar  with  no  line  skeleton,  and  the  default  50%  opacity  avatar  with  line 
skeleton.  The  color  scheme  of  the  line  skeleton  has  been  modified  to  match  the  online  quiz  developed 
by  OGAL.  The  Show/Flide  Skeleton  button  will  add  or  remove  a  fully  opaque  avatar  with  no  line  skeleton 
next  to  the  current  avatar.  The  avatar  produced  by  this  button  will  always  play  the  control  MoCap  file. 
The  Left/Right  and  Front/Back  Camera  View  buttons  will  display  the  avatar(s)  facing  the  corresponding 
direction.  Free  Camera  View,  meaning  the  user  is  able  to  click  and  drag  the  camera  around,  can  be 
applied  through  the  button  itself  or  by  clicking  on  the  background.  The  Camera  Follow  On/Off  button 
can  be  clicked  to  hold  the  camera  stationary  or  fix  it  to  follow  the  main  avatar.  This  functionality  of  this 
button  is  disabled,  however,  when  the  camera  is  not  in  free  camera  mode.  The  last  button,  MoCap 
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Offsets,  displays  a  separate  window  containing  a  list  of  all  of  the  joints  of  the  avatar  and  boxes 
containing  their  current  offset  value,  which  is  displayed  in  Figure  12.  The  joints  and  values  are  all 
generated  through  an  XML  file.  These  values  can  be  altered  and  applied  to  the  specified  joint  of  the 
main  avatar  via  this  window. 


Figure  12:  Any  joint  offset  of  the  main  avatar  can  be  modified  through  the  MoCap  Offsets  window. 

After  viewing  the  online  quiz  developed  by  OGAL,  substantial  changes  were  made  to  the  layout  and 
capabilities  of  the  quiz  window  to  match  the  format  used.  In  the  top  left  corner  of  the  quiz  window, 
there  are  buttons  for  navigating  between  previous  or  next  questions  and  a  replay  button.  The  bottom 
right  corner  of  the  quiz  window  contains  a  Submit/Continue  button,  and  the  bottom  left  corner  displays 
a  quiz  completion  progress  bar.  Upon  the  expansion  of  the  quiz  window,  the  user  may  begin  the  quiz 
after  reading  through  the  instructions  on  the  introductory  page.  The  quiz  is  XML  generated,  and  it  is 
relatively  simple  for  the  developer  to  modify  various  properties  of  each  question.  A  major  feature  that 
has  been  added  includes  an  option  to  auto-play  a  selected  MoCap  file  at  the  beginning  of  each  question. 
The  motion  will  be  played  once  with  a  view  of  the  frontal  plane,  followed  immediately  by  the  same 


13 


motion  viewed  in  the  sagittal  plane.  Clicking  the  replay  button  at  the  end  of  auto-play  will  replay  the 
auto-played  motion,  while  clicking  the  regular  play  button  on  the  main  toolbar  will  play  the  last  selected 
MoCap  file.  The  question  format  can  be  either  multiple  choice  (with  or  without  an  additional 
question/options)  or  a  checklist,  as  shown  in  Figure  13. 


Prosthetic  Gait  Learning  Tool  !  ca  |!  0  1 .22 


Figure  13:  A  question  in  the  current  quiz  posed  using  a  checklist  format. 

The  user  is  restricted  from  moving  past  the  current  question  until  an  answer  has  been  submitted,  but 
once  an  answer  has  been  submitted,  a  feedback  page  is  displayed.  The  user's  response(s),  as  well  as  the 
correct  answer(s),  are  displayed.  If  incorrect,  the  user's  response  is  shown  in  red  text  next  to  a  red  X 
icon.  If  the  user  answers  correctly,  a  green  icon  with  a  check  mark  is  displayed.  Pictures  of  the  ideal  gait 
during  initial  contact,  mid  stance,  terminal  stance,  and  mid  swing  are  also  provided  on  the  feedback 
page;  an  example  is  shown  in  Figure  14.  Furthermore,  the  main  avatar  is  auto-played  with  the  motion 
corresponding  to  that  question,  if  one  exists,  with  the  control  avatar  playing  next  to  it.  The  quiz 
developer  can  also  specify  if  the  feedback  auto-play  should  be  viewed  using  only  the  frontal  or  sagittal 
plane  instead  of  the  default  showing  the  motion  in  both  planes. 
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Figure  14:  Quiz  feedback  page  based  on  the  user's  response  of  one  correct  checkbox  and  one  incorrect  checkbox  marked. 


Discussion  and  Conclusion 

Most  of  the  time  and  effort  for  MPL  was  focused  on  getting  the  IRB.  The  team  had  regular  discussions 
during  this  year  where  the  progress  with  various  aspects  of  the  project  was  evaluated  and  next  steps 
planned.  However,  a  lot  of  resistance  to  progress  was  felt  due  to  the  lack  of  IRB  and  actual  data  to 
visualize  the  tutorials.  In  the  absence  of  such  data,  mock  up  data  was  created  to  make  some  progress  on 
the  gait  deviations  ratings  development  and  the  software  development.  Now  that  the  team  has  IRB 
approval,  it  cannot  wait  to  make  further  progress  on  all  fronts  with  an  approval  of  a  No  Cost  Extension 
request.  The  development  version  of  the  software  tool  has  been  deployed  at  the  OGAL  lab  to  prove  the 
capability  of  distribution  of  the  software  when  it  is  ready. 


When  the  tool  is  fully  developed,  it  will  provide  practitioners  at  all  clinics  and  hospitals  with  access  to 
advanced,  computer-based  gait  analysis  tools  that  are  currently  available  at  only  a  few  state-of-the-art 
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gait  laboratories.  The  tool  will  aid  in  the  training  of  service  providers,  ultimately  improving  the  level  of 
care  they  provide  to  wounded  veterans.  This  system  will,  thus,  provide  a  comprehensive  training 
experience,  allowing  practitioners  to  benefit  from  a  broad  array  of  patient  data  previously  collected  by 
the  US  Army,  thus  bridging  a  critical  gap  in  current  medical  training  practices.  The  system  will  be 
developed  to  accommodate  additional  sequences  captured  over  time,  thus  offering  an  extensible, 
distributable,  and  sustainable  training  library. 
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